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(54) ABNORMALITY DETECTING AND MONITORING SYSTEM 
(57)Abstract: 

PROBLEM TO BE SOLVED: To provide an abnormality detecting and monitoring 
system changing an operation condition in accordance with an image pickup 
environment change, giving an abnormality detection notice, judging the impossibility 
of abnormality detection and outputting a detection impossibility notice through the 
use of a first sensor which image-picks up a monitoring object and a second sensor 
detecting it with a means except for image pickup. 

SOLUTION: The abnormality detecting and monitoring system is constituted of a first 
sensor 2 which image-picks up a monitoring object, a second sensor 3 detecting the 
monitoring object by a means except for image pick up, a first sensor part 4 detecting 
image data in a monitoring image area, a first abnormality detecting part 10 detecting 
an abnormal phenomenon and outputting abnormality data, a second sensor part 5 
converting a sensor signal inputted from the second sensor 3 into sensor data and 



outputting it, a second abnormality detecting part 30 detecting the abnormal 
phenomenon from sensor data and outputting abnormality data, and an abnormality 
detection judgment part 50 comparing and operating abnormality data from the first 
abnormality detection part 10 with abnormality data from the second abnormality 
detection part 30 and giving an abnormality notice. 

* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect 
the original precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]The first sensor, such as a television camera which picturizes a surveillance 
object, and the second sensor, such as an infrared sensor which detect a surveillance 
object by means other than an image pick-up, and carries out the detect output of the 
sensor signal, Set a surveillance image section as a picture inputted from said first 
sensor, detect an abnormal phenomenon of an applicable image section on the first 
sensor part that detects image data of this image section, and conditions beforehand 
set up based on said image data, and output the first abnormal data, and. The first 
malfunction detection part which outputs error data when image data of an applicable 
image section is not within the limits set up beforehand, Detect an abnormal 
phenomenon of an applicable sensor on the second sensor part that carries out the 
conversion output of the sensor signal inputted from said second sensor to sensor 
data, and conditions beforehand set up based on said sensor data, and. When a sensor 
signal of an applicable sensor is not within the limits set up beforehand, carry out the 
comparison operation of the second malfunction detection part which outputs error 
data, and abnormal data from said first malfunction detection part and abnormal data 
from said second malfunction detection part, perform an abnormality notice, and. 
Carry out the comparison operation of the error data from said first malfunction 
detection part and the second malfunction detection part to judgement data set up 
beforehand, and it constitutes from a malfunction detection monitoring instrument 



which has an abnormality judgment part which outputs a detection improper notice of 
said sensor, Detection of an abnormal phenomenon of a surveillance object performs 
sensor data from the second sensor part to a subject as assistance, and image data 
from the first sensor part. A malfunction detection supervising system when said 
image data or said sensor data is not within the limits set up beforehand, wherein said 
malfunction detection judgment part outputs an abnormality notice based on data 
from a sensor of what one or outputs a detection improper notice. 
[Claim 2]The first sensor part and the first comparison-operation part which carries 
out the comparison operation of said image data characterized by comprising the 
following, The first computing equation selecting part which sets up 
comparison-operation types, such as an upper limit comparison operation and a lower 
limit comparison operation, The first threshold memory which has memorized a 
threshold of said comparison operation, and the first threshold selecting part which 
chooses a threshold used for said operation, The first malfunction detection part 
which has the first abnormality determining part that performs a predetermined 
abnormality judgement operation of an applicable image section based on the result of 
an operation of said first comparison-operation part, and performs which abnormality 
judgement for which normality, an unusual division, and detection are improper, A 
sensor selecting part which chooses a sensor signal inputted from said two or more 
second sensors, A/D (Analog Digital converter) which carries out the conversion 
output of said selected sensor signal to sensor data of digital signal form, The second 
sensor part that has a sensor data memory which stores this sensor data temporarily, 
The second comparison-operation part which performs a comparison operation based 
on said sensor data, and the second computing equation selecting part which chooses 
a computing equation which performs comparison operations, such as a level 
operation, The second operation condition set part which sets up an operation 
condition of said operation, and a level maximum for said comparison operation, a level 
minimum and the second threshold memory that has memorized which undetectable 
threshold, The second malfunction detection part which has the second threshold 
selecting part which chooses a threshold used for said comparison operation, and the 
second abnormality determining part that performs a predetermined abnormality 
judgement operation of an applicable image section based on the result of an 
operation of said second comparison-operation part, A malfunction detection 
judgment part which performs to a subject which abnormality notice for which 
normality, an unusual division, and detection are improper for an abnormality 
judgement result from said first malfunction detection part, Constitute from a 
malfunction detection judgment part which has a judgment mode set part which sets 
up malfunction detection judgment mode, and said malfunction detection judgment 
part, The malfunction detection supervising system according to claim 1 performing 
which abnormality notice for which normality, an unusual division, and detection are 



improper by independent judgment based on malfunction detection data from the first 

malfunction detection part or the second malfunction detection part. 

A/D which changes a picture signal from said first sensor into digital image data for 

said malfunction detection monitoring instrument (Analoge Digital converter). 

An image data memory which memorizes said digital image data. 

A surveillance image section set part which sets up a surveillance image section for 
malfunction detection. 

Operation function memories which have remembered a calculation program of said 
operation function, etc. to be an operation function set part which sets operation 
functions, such as an average value operation for said image data operation, and a 
peak value operation, to image data operation part which performs an image data 
operation of an applicable image section. 

[Claim 3]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the weighted-average-efficiency operation function which 
calculates a weighted average efficiency of image data within a predetermined image 
section to operation function memories of said first sensor part. 
[Claim 4]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the temporal change value operation function which 
calculates a temporal change value of image data within a predetermined image 
section to operation function memories of said first sensor part. 

[Claim 5]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the area value operation function which calculates an area 
value of image data within a predetermined image section to operation function 
memories of said first sensor part. 

[Claim 6]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the chromaticity value operation function which calculates a 
chromaticity value of image data within a predetermined image section to operation 
function memories of said first sensor part. 

[Claim 7]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the motion detection function which detects a motion of 
image data within a predetermined image section to operation function memories of 
said first sensor part. 

[Claim 8]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the shape detection function which detects shape of image 
data within a predetermined image section to operation function memories of said first 
sensor part. 

[Claim 9]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the duration time computing equation which calculates 
duration time of a predetermined threshold to the first computing equation selecting 



part of said first malfunction detection part. 

[Claim 10]The malfunction detection supervising system according to claim 2 carrying 
out additional memory of the number-of-times computing equation of a loop which 
calculates the number of times of a loop of a predetermined threshold to the first 
computing equation selecting part of said first malfunction detection part. 
[Claim 1 1]The malfunction detection supervising system according to claim 2 carrying 
out additional installation of the first operation condition set part which sets an 
operation condition of said first abnormality determining part as said first malfunction 
detection part, and enabling setting out of operation conditions, such as a luminosity 
of a surveillance object part, and lighting classification. 

[Claim 12]The malfunction detection supervising system according to claim 2 carrying 
out additional installation of the first operation condition set part which sets an 
operation condition of said first abnormality determining part as said first malfunction 
detection part, and enabling setting out of operation conditions, such as a surveillance 
object time zone. 

[Claim 13]The malfunction detection supervising system according to claim 2, wherein 
the second computing equation selecting part of said second malfunction detection 
part enables additional selection of a duration time computing equation which 
calculates duration time of a predetermined threshold. 

[Claim 14]The malfunction detection supervising system according to claim 2, wherein 
the second computing equation selecting part of said second malfunction detection 
part enables additional selection of a sensor number computing equation which 
calculates the sensor number of a predetermined threshold. 

[Claim 15]The malfunction detection supervising system according to claim 2, wherein 
the second computing equation selecting part of said second malfunction detection 
part enables additional selection of a motion computing equation which calculates a 
motion of a surveillance object from two or more sensor data. 

[Claim 16]The malfunction detection supervising system according to claim 2, wherein 
the second computing equation selecting part of said second malfunction detection 
part enables additional selection of a number-of-times computing equation which 
calculates appearance frequency of a predetermined threshold. 

[Claim 1 7]The malfunction detection supervising system according to claim 2, wherein 
the second computing equation selecting part of said second malfunction detection 
part enables additional selection of a period computation type which calculates an 
appearance cycle of a predetermined threshold. 

[Claim 18]The malfunction detection supervising system according to claim 2, wherein 
the second threshold selecting part of said second malfunction detection part enables 
additional selection of an appearance period value of a predetermined threshold. 
[Claim 19]The malfunction detection supervising system according to claim 2, wherein 
the second threshold selecting part of said second malfunction detection part enables 



additional selection of a periodicity variation of a predetermined threshold. 
[Claim 20]The malfunction detection supervising system according to claim 2 carrying 
out additional installation of the coincidence judgment mode in which coincidence of 
malfunction detection data from the first malfunction detection part and the second 
malfunction detection part performs which abnormality notice for which normality, an 
unusual division, and detection are improper to said judgment mode set part. 
[Claim 21]The malfunction detection supervising system according to claim 2 carrying 
out additional installation of the priority judgment mode in which which abnormality 
notice for which normality, an unusual division, and detection are improper is 
performed, with malfunction detection data which gave priority over said judgment 
mode set part to any of the first malfunction detection part or the second malfunction 
detection part they are. 

[Claim 22]The malfunction detection supervising system according to claim 2 carrying 
out additional installation of the comparative judgment mode in which carry out the 
comparison operation of the malfunction detection data of the first malfunction 
detection part or the second malfunction detection part to said judgment mode set 
part, and which abnormality notice for which normality, an unusual division, and 
detection are improper is performed to it. 

[Claim 23]The malfunction detection supervising system according to claim 2 carrying 
out additional installation of the detection data mode which carries out the direct 
output of said monitoring data or the sensor data to an output of said malfunction 
detection judgment part. 

[Claim 24]The comparison operation of the malfunction detection data of the first 
malfunction detection part or the second malfunction detection part is carried out to 
an output of said malfunction detection judgment part, The malfunction detection 
supervising system according to claim 2 carrying out additional installation of the 
detection mode change output which takes out detection mode change instructions to 
said monitor condition set part when it is preferred to change detection mode and to 
perform a re-judging. 

[Claim 25]The malfunction detection supervising system according to claim 1 using 
said second sensor as an ultrasonic sensor which detects an invader and intruding 
objects using an ultrasonic wave. 

[Claim 26]The malfunction detection supervising system according to claim 1 using 
said second sensor as an infrared array sensor which detects an invader, intruding 
objects, or an infrared emission object using infrared rays or a near-infrared sensor 
array. 

[Claim 27]The malfunction detection supervising system according to claim 1 using 
said second sensor as an infrared camera which carries out abnormality detecting 
from an infrared image. 

[Claim 28]The malfunction detection supervising system according to claim 1 using 



said second sensor as a collecting microphone which carries out abnormality 
detecting with a breaking sound, a sound, etc. 

[Claim 29]The malfunction detection supervising system according to claim 1 carrying 
out additional installation of the surveillance program memory which sets up 
abnormality monitor mode for every time, and setting supervision mode automatically 
as said malfunction detection device for every time and sensor. 

[Claim 30]The malfunction detection supervising system according to claim 1 which 
carries out carrying out additional installation of the image synthesis section which 
superimposes said abnormality notice data, sensor data, etc. on said input picture 
signal to said malfunction detection device with the feature. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the malfunction detection supervising 
system which notifies a detection failure, when it is concerned with the malfunction 
detection supervising system which used the television camera, and the detection 
notice of the abnormal circumstances is given using the second sensor detected by 
the image sensor which picturizes especially a surveillance object, and means other 
than an image pick-up and detection disabling occurs. 
[0002] 

[Description of the Prior Artl Drawing 3 is an outline system chart of the abnormality 
monitoring system by conventional technology. This figure and a part of drawing 2 are 
diverted, and the composition of the abnormality monitoring system by conventional 
technology, operation, and a technical problem are explained. The second sensor 3 
and 3, such as an infrared sensor which an abnormality monitoring system detects a 
surveillance object as the first sensor 2 that picturizes a surveillance object by means 
other than an image pick-up, and carries out the detect output of the sensor signal. 
The first sensor part 4 that sets a surveillance image section as the picture inputted 
from the first sensor 2, and detects the image data of this image section, The first 
malfunction detection part 10 which detects the abnormal phenomenon of an 
applicable image section on the conditions beforehand set up based on image data, 
and outputs abnormal data, It comprises the second sensor part 5 that carries out the 
conversion output of the sensor signal inputted from the second sensor to sensor 
data, the second malfunction detection part 30 which detects the abnormal 
phenomenon of an applicable sensor on the conditions beforehand set up based on 
sensor data, and the image synthesis section 6 which carries out superposition 



composition of the abnormality notice at a surveillance display image. 
[0003]As for the first sensor 2 that picturizes a surveillance object, the television 
camera is usually used. The video signal of television cameras is used as the 
monitoring device for surveillance generating picture 9 via the image synthesis section 
6, and it is inputted into the first sensor part 4. The first sensor part 4 changes an 
analog video signal into digital image data by A/D41, and stores it temporarily at the 
image data memory 42. The surveillance image section set part 43 sets up the 
surveillance image section of a surveillance object, from the image data memory 42, 
reads image data and inputs it into the image data operation part 44. The image data 
operation part 44 carries out the operation output of the image data based on the 
operation function beforehand memorized by the operation function memories 48. The 
operation function set part 45 sets up a predetermined operation function out of two 
or more operation functions. The image data by which the operation output was 
carried out is inputted into the first malfunction detection part 10. The surveillance 
data memory 11 is made to once memorize the first malfunction detection part 10 by 
using this image data as monitoring data. The first comparison-operation part 12 
carries out the comparison operation of the threshold data by which memory 
registration is carried out beforehand to the monitoring data read from the 
surveillance data memory 1 1, and the first threshold memory 15, and inputs the result 
of an operation into the first abnormality determining part 17. If the first abnormality 
determining part 17 judges that a comparison-operation result is equivalent to 
threshold data, or is over threshold data, it will judge with an abnormal occurrence and 
will take out an abnormality notice. The kind data and abnormal occurrence image 
section data of a computing equation are contained in an abnormality notice. 
[0004]It is an auxiliary sensor which reacts when an infrared sensor etc. are generally 
used in the second sensor 3 and 3, and people cross the installed place on the other 
hand or an unusual elevated temperature occurs. 
In many cases, plurality is installed. 

In the second sensor part 5, the sensor signal from two or more second sensors 3 and 
3 is changed into digital sensor data, and is inputted into the second malfunction 
detection part 30. The second malfunction detection part 30 carries out a comparison 
operation to the threshold set up beforehand, and if it judges that equivalent to 
threshold data or it is over threshold data, it will output an abnormality notice. This 
abnormality notice is used for switching the recording mode of Time Rappus VTR 
which is not illustrated to a real time mode etc. from time RAPPUSU mode etc. This 
abnormality notice may be superimposed on monitor images by the image synthesis 
section 6. 

[0005]In the abnormality monitoring system by conventional technology, the first 
sensor part 4 and the second sensor part 5 are the independent subsystems, and it 
does not have a role which improves the reliability of a mutual supplement or 



malfunction detection. From image data, the first sensor part 4 sets up a surveillance 
object image section, and is carrying out the comparison operation of the image data. 
In business-hours inner and outside, the image pick-up environment of a surveillance 
object differs in visitor a large number, uninhabited one, etc. substantially, and needs 
to change the above-mentioned operation function and operation condition of a 
comparison operation in the - night daytime. Then, the thing for which the first sensor 
part and the second sensor part are combined organically, and an operation condition 
is changed corresponding to the image pick-up environmental variation of a 
surveillance object, Technical problems, such as making the first sensor part and the 
second sensor part into complementary relationship and issuing the notice of an or 
[ detection of abnormalities is performed correctly (is neither a mischief nor 
disturbance received?) ], occur, and the proposal of the malfunction detection 
supervising system corresponding to this is called for. 
[0006] 

[Problem(s) to be Solved by the Invention]In view of the technical problem explained 
above, this invention changes an operation condition corresponding to an image 
pick-up environmental variation using the first sensor that picturizes a surveillance 
object, and the second sensor that detects a surveillance object by means other than 
an image pick-up, and perform the notice of malfunction detection, and. It aims at the 
proposal of the malfunction detection supervising system which makes an improper 
judgment of malfunction detection and outputs a detection improper notice. 
[0007] 

[Means for Solving the Problem]The first sensor, such as a television camera which 
picturizes a surveillance object, and the second sensor, such as an infrared sensor 
which detect a surveillance object by means other than an image pick-up, and carries 
out the detect output of the sensor signal, Set a surveillance image section as a 
picture inputted from the first sensor, detect an abnormal phenomenon of an 
applicable image section on the first sensor part that detects image data of this image 
section, and conditions beforehand set up based on image data, and output the first 
abnormal data, and. The first malfunction detection part which outputs error data 
when image data of an applicable image section is not within the limits set up 
beforehand, Detect an abnormal phenomenon of an applicable sensor on the second 
sensor part that carries out the conversion output of the sensor signal inputted from 
the second sensor to sensor data, and conditions beforehand set up based on sensor 
data, and. When a sensor signal of an applicable sensor is not within the limits set up 
beforehand, carry out the comparison operation of the second malfunction detection 
part which outputs error data, and abnormal data from the first malfunction detection 
part and abnormal data from the second malfunction detection part, perform an 
abnormality notice, and. The comparison operation of the error data from the first 
malfunction detection part and the second malfunction detection part is carried out to 



judgement data set up beforehand, and it constitutes from a malfunction detection 
monitoring instrument which has a malfunction detection judgment part which outputs 
a detection improper notice of a sensor. 

[0008]A/D which changes a picture signal from the first sensor into digital image data 
for said malfunction detection monitoring instrument, An image data memory which 
memorizes digital image data, and a surveillance image section set part which sets up 
a surveillance image section for malfunction detection, An operation function set part 
which sets operation functions, such as an average value operation for an image data 
operation, and a peak value operation, to image data operation part which performs an 
image data operation of an applicable image section, The first sensor part that has 
operation function memories which have memorized a calculation program of an 
operation function, etc., The first comparison-operation part which carries out the 
comparison operation of the image data, and the first computing equation selecting 
part which sets up comparison-operation types, such as an upper limit comparison 
operation and a lower limit comparison operation, The first threshold memory which 
has memorized a threshold of a comparison operation, and the first threshold 
selecting part which chooses a threshold used for an operation, The first malfunction 
detection part which has the first abnormality determining part that performs a 
predetermined abnormality judgement operation of an applicable image section based 
on the result of an operation of the first comparison-operation part, and performs 
which abnormality judgement for which normality, an unusual division, and detection 
are improper, The second sensor part that has a sensor selecting part which chooses 
a sensor signal inputted from said two or more second sensors, A/D which carries out 
the conversion output of the selected sensor signal to sensor data of digital signal 
form, and a sensor data memory which stores this sensor data temporarily, The 
second comparison-operation part which performs a comparison operation based on 
sensor data, and the second computing equation selecting part which chooses a 
computing equation which performs comparison operations, such as a level operation, 
The second operation condition set part which sets up an operation condition of an 
operation, and a level maximum for a comparison operation, a level minimum and the 
second threshold memory that has memorized which undetectable threshold, The 
second malfunction detection part which has the second threshold selecting part 
which chooses a threshold used for a comparison operation, and the second 
abnormality determining part that performs a predetermined abnormality judgement 
operation of an applicable image section based on the result of an operation of the 
second comparison-operation part, It constitutes from a malfunction detection 
judgment part which has a malfunction detection judgment part which performs to a 
subject which abnormality notice for which normality, an unusual division, and 
detection are improper for an abnormality judgement result from the first malfunction 
detection part, and a judgment mode set part which sets up malfunction detection 



judgment mode. 

[0009]. Carry out additional memory of the weighted-average-efficiency operation 
function which calculates a weighted average efficiency of image data within a 
predetermined image section to operation function memories of the first sensor part. . 
Carry out additional memory of the temporal change value operation function which 
calculates a temporal change value of image data within a predetermined image 
section. . Carry out additional memory of the area value operation function which 
calculates an area value of image data within a predetermined image section. 
Additional memory of the shape detection function which carries out additional 
memory of the motion detection function which carries out additional memory of the 
chromaticity value operation function which calculates a chromaticity value of image 
data within a predetermined image section, and which detects a motion of image data 
within a predetermined image section, or detects shape of image data within a 
predetermined image section is carried out. 

[0010]Additional memory of the number-of-times computing equation of a loop which 
carries out additional memory of the duration time computing equation which 
calculates duration time of a predetermined threshold, or calculates the number of 
times of a loop of a predetermined threshold to the first computing equation selecting 
part of the first malfunction detection part is carried out. 

[001 1]additional installation of the first operation condition set part which sets an 
operation condition of the first abnormality determining part as the first malfunction 
detection part being carried out, and setting out of operation conditions, such as a 
luminosity of a surveillance object part and lighting classification, being enabled, or, 
Additional installation of the first operation condition set part which sets up an 
operation condition of the first abnormality determining part is carried out, and setting 
out of operation conditions, such as a surveillance object time zone, is enabled. 
[0012]. The second computing equation selecting part of the second malfunction 
detection part enables additional selection of a duration time computing equation 
which calculates duration time of a predetermined threshold. . Enable additional 
selection of a sensor number computing equation which calculates the sensor number 
of a predetermined threshold. . Enable additional selection of a motion computing 
equation which calculates a motion of a surveillance object from two or more sensor 
data. To enable additional selection of an appearance period value of a predetermined 
threshold which enables additional selection of a period computation type which 
calculates an appearance cycle of a predetermined threshold which enables additional 
selection of a number-of-times computing equation which calculates appearance 
frequency of a predetermined threshold, or additional selection of a periodicity 
variation of a predetermined threshold is enabled. 

[0013]To a judgment mode set part, by coincidence of malfunction detection data 
from the first malfunction detection part and the second malfunction detection part, 



normality, . Carry out additional installation of the coincidence judgment mode in 
which which abnormality notice for which an unusual division and detection are 
improper is performed. . Carry out additional installation of the priority judgment mode 
in which malfunction detection data over which priority was given to any of the first 
malfunction detection part or the second malfunction detection part they are 
performs which abnormality notice for which normality, an unusual division, and 
detection are improper. Or additional installation of the comparative judgment mode in 
which carry out the comparison operation of the malfunction detection data of the 
first malfunction detection part or the second malfunction detection part, and which 
abnormality notice for which normality, an unusual division, and detection are improper 
is performed is carried out. 

[0014]. Carry out additional installation of the detection data mode which carries out 
the direct output of monitoring data or the sensor data to an output of a malfunction 
detection judgment part. Or the comparison operation of the malfunction detection 
data of the first malfunction detection part or the second malfunction detection part 
is carried out, and when it is preferred to change detection mode and to perform a 
re-judging, additional installation of the detection mode change output which takes 
out detection mode change instructions to a monitor condition set part is carried out. 
[0015]. Use the second sensor as an ultrasonic sensor which detects an invader and 
intruding objects using an ultrasonic wave. It is considered as a collecting microphone 
which is used as an infrared camera which is used as an infrared array sensor which 
detects an invader, intruding objects, or an infrared emission object using infrared rays 
or a near-infrared sensor array, and which carries out abnormality detecting from an 
infrared image, or carries out abnormality detecting with a breaking sound, a sound, 
etc. 

[0016]Furthermore additional installation of the surveillance program memory which 
sets abnormality monitor mode for every time as a malfunction detection device is 
carried out, or additional installation of the image synthesis section which 
superimposes abnormality notice data, sensor data, etc. on an input picture signal is 
carried out. 
[0017] 

[Embodiment of the Invention] Drawing 1 is an outline system chart of one example of 
the malfunction detection supervising system by this invention. Drawing 2 is an 
important section block diagram of one example of the malfunction detection 
monitoring instrument by this invention. The outline system configuration of a 
malfunction detection supervising system is explained using drawing 1 . The first 
sensor 2, such as a television camera with which the malfunction detection 
supervising system of an example picturizes a surveillance object, The second sensor 
3 and 3, such as an infrared sensor which detect a surveillance object by means other 
than an image pick-up, and carries out the detect output of the sensor signal. Set a 



surveillance image section as the picture inputted from the first sensor 3, detect the 
abnormal phenomenon of an applicable image section on the first sensor part 4 that 
detects the image data of this image section, and the conditions beforehand set up 
based on image data, and output the first abnormal data, and. The first malfunction 
detection part 10 which outputs error data when the image data of an applicable image 
section is not within the limits set up beforehand, and the sensor signal inputted from 
the second sensor 3 on the second sensor part 5 that carries out a conversion output 
to sensor data, and the conditions beforehand set up based on sensor data the 
abnormal phenomenon of an applicable sensor. When it detects and the sensor signal 
of an applicable sensor is not within the limits set up beforehand, carry out the 
comparison operation of the abnormal data from the second malfunction detection 
part 30 and the first malfunction detection part 10 which output error data, and the 
abnormal data from the second malfunction detection part 30, perform an abnormality 
notice, and. The comparison operation of the error data from the first malfunction 
detection part 10 and the second malfunction detection part 30 is carried out to the 
judgement data set up beforehand, It comprises the malfunction detection monitoring 
instrument 1 etc. which have the abnormality judgment part 50 which outputs the 
detection improper notice of a sensor. 

[0018]Comprehensive operation of the detailed composition of a malfunction 
detection monitoring instrument and the system of an example is explained using 
drawing 2 . The malfunction detection monitoring instrument 1 of an example, The 
picture signal from the first sensor 2. The average value operation for the surveillance 
image section set part 43 which sets up the surveillance image section for A/D41 
changed into digital image data, the image data memory 42 which memorizes digital 
image data, and malfunction detection, the image data operation part 44 which 
performs the image data operation of an applicable image section, and an image data 
operation, The first sensor part 4 that has the operation function memories 46 etc. 
which have memorized the calculation program etc. of the operation function set part 
45 which sets up operation functions, such as a peak value operation, and an 
operation function, The first comparison-operation part 12, upper limit comparison 
operation which carry out the comparison operation of the image data, Based on the 
result of an operation of the first computing equation selecting part 1 3 which sets up 
comparison-operation types, such as a lower limit comparison operation, the first 
threshold memory 15 which has memorized the threshold of a comparison operation, 
the first threshold selecting part 14 which chooses the threshold used for an 
operation, and the first comparison-operation part, perform the predetermined 
abnormality judgement operation of an applicable image section, and Normality, an 
unusual division, The first malfunction detection part 10 which has the first 
abnormality determining part 17 etc. that perform which abnormality judgement [ being 
undetectable ], the sensor selecting part 5a which chooses the sensor signal inputted 



in said two or more second sensors 3 and 3, A/D 5b which carries out the conversion 
output of the selected sensor signal to the sensor data of digital signal form, The 
second sensor part 5 that has the sensor data memory 5c etc. which store this sensor 
data temporarily, The level maximum for the second computing equation selecting part 
33 which chooses the computing equation which performs comparison operations, 
such as the second comparison-operation part 31 which performs a comparison 
operation based on sensor data, and a level operation, the second operation condition 
set part 32 which sets up the operation condition of an operation, and a comparison 
operation, A level minimum, the second threshold memory 35 which has memorized 
which undetectable threshold, the second threshold selecting part 34 which chooses 
the threshold used for a comparison operation, and the second malfunction detection 
part 30 which has the second abnormality determining part 37 that performs the 
predetermined abnormality judgement operation of an applicable image section based 
on the result of an operation of the second comparison-operation part 31, the 
malfunction detection judgment part which has the malfunction detection judgment 
part 51 which performs to a subject which abnormality notice for which normality, an 
unusual division, and detection are improper for the abnormality judgement result from 
the first malfunction detection part 17, the judgment mode set part 52 which sets up 
malfunction detection judgment mode, etc. — 50 — the standard configuration is 
carried out. 

[0019]Detailed operation is explained along with the data flow from each sensor. The 
video signal outputted from the first sensor 2 (television camera) that picturizes a 
surveillance object is used as the monitoring device for surveillance generating picture 
9 via the image synthesis section 6, and it is inputted into the first sensor part 4. The 
first sensor part 4 changes an analog video signal into digital image data by A/D41, 
and stores it temporarily at the image data memory 42. With the device of an example, 
from 192 image sections, the image section of vertical (12) x width (16) =192 is 
provided in the picturized picture, and the surveillance image section set part 43 sets 
up the surveillance image section of a surveillance object, it reads image data from the 
image data memory 42, and inputs it into the image data operation part 44. The image 
data operation part 44 carries out the operation output of the image data based on the 
operation function beforehand memorized by the operation function memories 48. The 
operation function set part 45 sets up a predetermined operation function out of two 
or more operation functions. 

[0020]Since the candidates for surveillance detection differ for every image sections, 
such as an invader's detection, detection of invasion vehicles, theft detection, 
destructive detection, and fire detection, the operation function beforehand 
memorized by the operation function memories 48, The peak value operation which 
catches the average value operation for which a surveillance object detects the 
luminosity of an image section, a sudden flash, etc., Within the 



weighted-average-efficiency operation which detects the level weight averaged 
according to the gradation of a luminosity, the area value operation which defines the 
threshold of image levels and calculates the area, the chromaticity value operation 
which calculates the chromaticity value within an image section, the same image 
section, or a contiguity image section, It is a shape value operation etc. which 
recognize the motion detection which does the detection operation of the movement 
of specific gradation with time progress, and the shape set up beforehand. The image 
data by which the operation output was carried out is inputted into the first 
malfunction detection part 10. 

[0021 ]It is inputted and image data is once memorized by the surveillance data 
memory 1 1 as monitoring data. The first comparison-operation part 12 carries out the 
comparison operation of the threshold data by which memory registration is carried 
out beforehand to the monitoring data read from the surveillance data memory 1 1 , and 
the first threshold memory 15 using the computing equation which the first computing 
equation selecting part 13 chose, and inputs the result of an operation into the first 
abnormality determining part 17. The first computing equation selecting part 13 a 
selectable computing equation, It is selectable in the lower limit comparison-operation 
type which detects an operation stop, image pick-up disturbance, etc. of the upper 
limit comparison-operation type of the monitoring data showing temperature, a 
breaking sound, etc. and a camera, the duration time comparison-operation type 
which carries out the comparison operation of the duration time of a predetermined 
threshold, the number-of-times computing equation of a loop which calculates the 
number of times of a loop of a predetermined threshold, etc. 

[0022]The comparison operation of the first abnormality determining part 1 7 is carried 
out by the operation condition which the first operation condition set part 1 6 set up, 
and if it judges that a comparison-operation result is equivalent to threshold data, or 
is over threshold data, it will judge with an abnormal occurrence and will take out an 
abnormality notice. The kind data and abnormal occurrence image section data of a 
computing equation are contained in an abnormality notice. The operation condition 
which the first operation condition set part 16 sets up, There are luminosities 
(daytime, night, and direct sunlight etc.) of image pick-up environment, lighting 
classification (usually interior illumination, night-light, and non-visible light lighting 
etc.), a time zone (a business-hours belt, a closing hour belt, the belt between 
non-man-hour), etc., it uses together with nonrandom assortment, such as the 
above-mentioned image section setting out, operation function setting out, and 
threshold setting, and an operation condition is set up. 

[0023]Subsequently, the data flow from the second sensor 3 is explained. The second 
sensor 3 is installed in order to complement the first sensor 2, such as a television 
camera which picturizes a surveillance object, and it detects a surveillance object by 
means other than an image pick-up. Infrared sensors, such as an infrared photo 



detector which detects infrared catoptric light or discharge light in the system of an 
example, Using the ultrasonic sensor, the infrared rays, or the near-infrared sensor 
array which detects an invader and intruding objects using an ultrasonic wave, an 
invader, The collecting microphone etc. which carry out abnormality detecting with 
the infrared camera and breaking sound which carry out abnormality detecting, a 
sound, etc. are used in combination by the surveillance object of a setting position or 
a surveillance image section from intruding objects or the infrared array sensor which 
detects an infrared emission object, and the infrared image. These sensors can 
incorporate an abnormality notice into the inside of a device from the interface part 
which is not illustrated in this system, although there are some which perform an 
abnormality notice as single operation. If in charge of installation, it is installed in the 
position corresponding to the surveillance image section of the first sensor 2, and a 
setting position or a detection place, and sensor classification are beforehand 
registered from the installation image section setting input part which is not illustrated. 
[0024]In the second sensor part 5, the sensor selecting part 5a carries out the 
selection input of the sensor signal from the second predetermined sensor 3 to A/D 
5b. A/D 5b changes a sensor signal into digital sensor data, and is made to store it 
temporarily with installation image section data and sensor kind data at a sensor data 
memory. 

[0025]The second comparison-operation part 31 carries out a comparison operation 
to the threshold which has memorized the inputted sensor data beforehand in the 
second threshold memory 35 using the computing equation which the second 
computing equation selecting part 33 chose. The second computing equation selecting 
part 33 selection a possible computing equation, The level computing equation which 
calculates sensor output levels, such as a luminosity, the duration time computing 
equation which calculates the duration time of a predetermined threshold, The sensor 
number computing equation which calculates the sensor number of a predetermined 
threshold, the motion computing equation which calculates a motion of a surveillance 
object from two or more sensor data, They are a number-of-times computing 
equation which calculates the appearance frequency of a predetermined threshold, a 
period computation type which calculates the appearance cycle of a predetermined 
threshold, an appearance period value computing equation of a predetermined 
threshold, a periodicity change computing equation of a predetermined threshold, etc. 
[0026]The threshold memorized beforehand is chosen as the second threshold 
memory 35 by the second threshold selecting part 34. The level upper limit as which 
the kind of selectable threshold detects a flame, generation of heat, a breaking sound, 
etc., The level lower limit which detects protection from light by an invader, the 
lighting interception by sabotage, or sensor part protection from light, It is a detection 
improper decision value etc. which detect detection disabling by the sensor period 
value which detects the cycle of periodic luminosity change of a neon sign, a revolving 



light, etc., the periodicity variation which detects a periodic variable element and the 
malfunction of a sensor, cutting of a connecting circuit, etc. 

[0027]The result of an operation of the second comparison-operation part 31 is once 
memorized by the data memory 36, and it is inputted into the second malfunction 
detection part 30. The second malfunction detection part 30 carries out a comparison 
operation to the threshold set up beforehand, and if it judges that equivalent to 
threshold data or it is over threshold data, it will output an abnormality notice. 
[0028]The malfunction detection judgment part 51 performs to a subject which 
abnormality notice for which normality, an unusual division, and detection are improper 
for the abnormality judgement result from the first malfunction detection part by the 
malfunction detection judgment mode which the judgment mode set part 52 set up. 
The malfunction detection judgment mode which can set up the judgment mode set 
part 52, The independent judgment mode by independent judgment of only the 
abnormality judgement result of the first malfunction detection part 10, The 
coincidence judgment mode in which the abnormality judgement result the first 
malfunction detection part 10 and whose second malfunction detection part 30 
corresponded performs unusual judgment, The priority judgment mode in which give 
priority to judgment of the method of what, and unusual judgment is performed when 
the abnormality judgement result of the first malfunction detection part 10 and the 
second malfunction detection part 30 is inharmonious, And it is in comparative 
judgment mode in which the abnormality notice registered into the table which carried 
out the comparison operation of the unusual result of an operation of the first 
malfunction detection part 10 and the second malfunction detection part 30, and was 
set up beforehand is performed etc. 

[0029]The output forms of the malfunction detection judgment part 51 are an 
abnormality notice (unusual classification, unusual image section, and sensor 
classification), a detection improper notice (improper phenomenon classification, an 
applicable image section, an applicable sensor classification), etc. These abnormality 
notice data is outputted from the abnormality notice terminal 7 and the detection 
improper notice terminal 8, and superposition composition is carried out by the image 
synthesis section 6 at monitor images, and it is displayed on the monitor for 
surveillance via the generating picture terminal 9 as a picture signal. 
[0030]The setting-out instructions of the monitor conditions, such as a selection 
condition of each part and setups, are carried out from the monitor condition set part 
53 at each part of the inside of a device. It is a day unit or a week unit, and the 
surveillance program memory 54 by which additional installation was carried out has 
memorized the monitoring program which sets up a monitor condition for every time, 
and the monitor condition set part 53 reads this monitoring program according to the 
date data which is not illustrated, and it sets up automatically the monitor condition of 
each part of the inside of a device. 



[0031 ]The detected information output which carries out the direct output of the 
sensor data is added to the output form of the malfunction detection judgment part 51. 
As a result of carrying out the comparison operation of the unusual result of an 
operation of the first malfunction detection part 10 and the second malfunction 
detection part 30, after performing a comparison-operation function, threshold 
selection, and a surveillance image section addition and making a change, when 
abnormality judgement is required again, the detection mode change output 51c is 
sent out to the monitor condition set part 53. 
[0032] 

[Effect of the Invention]This invention is carried out by the contents explained above, 
and does so the effect described below. The first sensor that picturizes a surveillance 
object, and the second sensor that detects a surveillance object by means other than 
an image pick-up, By constituting from a first malfunction detection part which 
detects the abnormal phenomenon of a surveillance image section and outputs 
abnormal data, and a second malfunction detection part which detects an abnormal 
phenomenon and outputs abnormal data from the sensor signal from the second 
sensor, Detection of the abnormal phenomenon of a surveillance object performs the 
sensor data from the second sensor part to a subject as assistance by the image data 
from the first sensor part, and. When image data or sensor data was not within the 
limits set up beforehand, the proposal of the malfunction detection supervising system 
which outputs an abnormality notice based on the data from the sensor of what one, 
or outputs a detection improper notice of a malfunction detection judgment part was 
completed. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is an outline system chart of one example of the malfunction detection 
supervising system by this invention. 

[Drawing 2] It is an important section block diagram of one example of the malfunction 
detection monitoring instrument by this invention. 

[Drawing 3] It is an outline system chart of the abnormality monitoring system by 
conventional technology. 
[Description of Notations] 

1 Malfunction detection monitoring instrument 

2 The first sensor 

The 3 and 3' second sensor 
4 The first sensor part 



5 The second sensor part 
5a Sensor selecting part 
5b, 41 A/D 

5c Sensor data memory 

6 Image synthesis section 

7 Abnormality notice terminal 

8 A detection improper notice terminal 

9 Generating picture terminal 

10 The first malfunction detection part 

1 1 Surveillance data memory 

12 The first comparison-operation part 

13 The first computing equation selecting part 

14 The first threshold selecting part 

15 The first threshold memory 

16 The first operation condition set part 

17 The first abnormality determining part 

30 The second malfunction detection part 

31 The second comparison-operation part 

32 The second operation condition set part 

33 The second computing equation selecting part 

34 The second threshold selecting part 

35 The second threshold memory 

36 Data memory 

37 The second abnormality judgment part 

50 Malfunction detection judgment part 

51 Malfunction detection judgment part 

52 Judgment mode set part 

53 Monitor condition set part 

54 Surveillance program memory 
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